Recent studies have found that a large percentage of new information is stored on magnetic storage media as opposed to other forms of storage. In addition, a growth of at least 30% of new information has been recently estimated for each new year. Information can be stored in file systems, on magnetic tape, and in databases. The most critical of information, such as stock market information, medical information, and financial information, is likely to be stored in databases which in turn store information in file systems. The file systems, in turn, are stored on permanent physical media such as a disk device.
One large challenge to storing information electronically is that there are many layers between the end user and the underlying physical media that stores the data. The physical media can cause corruption in data that is stored, and so can any of the layers between a user and physical media, including but not limited to the database server itself. For example, data can be corrupted in system memory from a memory error, in a disk device or controller from hardware or driver errors, or from other errors in an operating system, program, or utility on any system in a solution. Such corruption can cause inaccurate data to be sent to, retrieved from, and/or stored on the physical media.
There are many possible layers in a “solution stack” (all the layers of a system from an application to physical storage, including an I/O stack), including such layers as the application layer, server operating system layer, file system layer, interface protocol layer, peripheral driver layer, hardware/software controller layer, and disk layer. Some methods exist for preventing corruption in data provided to some of the layers in an I/O stack. However, some of the layers in a system are not addressed by existing methods of maintaining data integrity. There are industry standards such as T10 and proprietary implementations that aim to catch and correct data corruption in the I/O stack, but none link the actual database with the other higher layers in the solution such as an application layer. Database systems are the primary storage interface for the vast majority of critically important information. If the database or earlier layer corrupts the data, all of the data protection “below” the database (down to the physical storage media) will not detect or prevent the data corruption.
Accordingly, what is needed is a method and apparatus that extends data integrity and the detection of data corruption in an I/O stack up through a database layer and other system layers to the application end. The present invention addresses such a need.